Auto-triggered camera self-timer based on recognition of subject&#39;s presence in scene

ABSTRACT

An automated image capture mode of a camera in an electronic device insures that a particular subject appears in the captured image. An image of the particular subject, which may be the photographer, is initially captured. Subsequently, another image capture is automatically triggered when the same subject is detected within the camera&#39;s field of view. In one embodiment, a motion sensor within the device may be employed to begin a search for the subject, when the camera is subjected to a sudden movement. Other embodiments are also described and claimed.

BACKGROUND

1. Field

The invention relates generally to the field of photography and, moreparticularly, to an automated process for capturing an image thatincludes a particular person, such as the photographer.

2. Background

It is a common desire of photographers to be included in scenes thatthey photograph. A frequently used solution is to enlist the aid ofanother individual to take the photograph. This, of course, is notalways a feasible or convenient solution. Most cameras are equipped withself-timers that delay the shutter release for some period of time sothat the photographer can position the camera on a stable surface andthen move into the camera's field of view. Additionally, some camerasare equipped with a remote control so that the photographer can triggerthe shutter release after taking a desired position in front of thecamera. Another common solution that is frequently used with smallhandheld cameras is for the photographer to hold the camera in anoutstretched arm with the camera lens pointed in the direction of thephotographer. This approach is often unsuccessful since the camera'sviewfinder is typically not visible from the front of the camera andnumerous attempts may be required before an image is captured withoutthe photographer's face being cropped out of the scene.

SUMMARY

An embodiment of the present invention provides a camera with anautomated image capture mode for insuring that a particular subjectappears in the captured image. An image of the particular subject, whichmay be the photographer, is initially captured. Subsequently, anotherimage capture is automatically triggered when the same person isdetected within the camera's field of view.

In another embodiment of the invention, a motion sensor within thecamera is employed to begin a search for the person whose image waspreviously stored, when the camera is subjected to a sudden movement.

BRIEF DESCRIPTION OF THE DRAWINGS

The embodiments of the invention are illustrated by way of example andnot by way of limitation in the figures of the accompanying drawings inwhich like references indicate similar elements. It should be noted thatreferences to “an” or “one” embodiment of the invention in thisdisclosure are not necessarily to the same embodiment, and they mean atleast one.

FIG. 1 illustrates a camera device in which embodiments of the presentinvention may be practiced.

FIG. 2 illustrates another camera device in which embodiments of thepresent invention may be practiced.

FIG. 3 is a functional block diagram of a camera device in whichembodiments of the present invention may be practiced.

FIG. 4 is a functional flow diagram of an image capture process inaccordance with an embodiment of the present invention.

FIG. 5 is a functional flow diagram of an image capture process inaccordance with another embodiment of the present invention.

DETAILED DESCRIPTION

Throughout the description, for the purposes of explanation, numerousspecific details are set forth in order to provide a thoroughunderstanding of the present invention. It will be apparent, however, toone skilled in the art that the present invention may be practicedwithout some of these specific details. In other instances, well-knownstructures and devices are shown in block diagram form to avoidobscuring the underlying principles of embodiments of the presentinvention.

FIG. 1 illustrates a portable camera device 10 in which embodiments ofthe present invention may be practiced. The device 10 may be any one ofseveral different types of small consumer electronic devices that can beeasily held in the user's hands during normal use. In particular, device10 may be a multi-function smart phone device such as an iPhone™ deviceby Apple Inc. The device 10 in this case has an exterior front face inwhich there is a display screen 12 and an imaging lens 16 of afront-facing camera 14 (also referred to as a front camera or a camerathat is aimed out of the front face). A second, rear-facing camera (notshown) may also be included whose imaging lens faces out of the exteriorrear or back face of the housing of the device 10. In one embodiment,the front camera 14 is a lesser performing camera (e.g., has lower imagesensor pixel resolution) than the rear camera, and the rear face of thehousing of the device 10 has no display screen. Embodiments of thepresent invention may also be practiced in a dedicated personal digitalcamera device 20 illustrated in FIG. 2 which may have an opticalviewfinder and in other camera devices, including a tablet-like computersuch as an iPad™ device by Apple Inc., a notebook personal computer or adesktop personal computer, all of which may have a built-in digitalcamera.

A combined circuit schematic and block diagram of some relevantconstituent electronic components of a portable camera device 10 (alsoapplicable to camera device 20) is presented in FIG. 3. The digitalcamera functionality is obtained using a microelectronic image sensorarray 15 on which light from the scene to be captured has been gatheredby the imaging lens 16. A mechanical shutter (not shown) may be includedin front of the sensor array 15 to control the duration of exposure.Alternatively, an entirely electronic shutter may be implemented usingthe sensor array's reset and readout signals. The sensor array 15responds to the incident light and collects photo-generated charge orforms image signals during an integration interval (or exposureinterval) defined by the reset and readout signals. At the end of theintegration interval, the image signals are read (including beingdigitized) and transferred as a captured digital image or “exposure” toimage storage 17. This may be a “final” image or picture of the scene,which may be stored in a removable, user picture storage area (e.g., aremovable nonvolatile memory card.) The image storage 17 may includesufficient non-volatile memory suitable to store many digital imagescaptured by the sensor array 15.

A camera controller 18 (e.g., implemented as a combination of programmeddata processing components and hardwired logic circuitry) manages theprocess of capturing images. It does so by generating various commandsthat are signaled to the components of the camera device 10, in responseto a user of the device 10 actuating a shutter release button 19, totake an exposure. The button 19 may be a physical button on an outsidesurface of the housing of the device 10, or it may be a virtual buttondisplayed on a touch screen of the device 10. The camera may alsoinclude a digital viewfinder function that is also conducted by thecontroller 18. The digital viewfinder function lets the user see a lowerresolution version of the just-captured image on the display screen 12.The digital viewfinder function may also produce live video of the sceneusing a selected camera of the device 10; the user can thus preview thescene before actuating the shutter release button to take a picture.

In the case of a camera device 10 having both a front facing camera anda rear facing camera, the controller 18 may be further coupled to asecond, different image sensor array via separate reset and readoutsignal lines. The second array would also be coupled to share the imagestorage 17 and display screen 12. A different imaging lens would also beprovided for gathering light onto the second sensor array, where thelatter combination could be aimed at a substantially opposite directionas the array 15 and imaging lens 16.

A process for implementing an embodiment of the present invention isshown in FIG. 4. This process may also be conducted by the controller18, within the arrangement of FIG. 3 for instance. The process beginswith user selection of the automated image capture process at 30. Thismay be accomplished with a menu selection, by launching a specialpurpose application, by depressing a manual switch or by other meansknown to those skilled in the art. An image of the subject is thencaptured at 32. The subject may be the user of the camera, but could beany other person that the camera user wishes to include in asubsequently captured image. In the case of a device having bothfront-facing and rear-facing cameras, such as an iPhone™ 4 device byApple Inc., an image of the user may be conveniently captured using thefront-facing camera, e.g., while at the same time the user sees herselfin the display screen 12 by virtue of the digital viewfinder functionexecuting off of the front-facing camera. The captured image is thenexamined at 34 to ensure that the subject's facial features have beenadequately captured in the image. If not, the image capture process isrepeated. Once the existence of facial features in the captured imagehas been verified, the controller 18 may signal some form of feedback tothe camera user, letting the user know that the automated image captureprocess can continue.

Once an acceptable image of the subject's face has been captured, theautomated picture taking process continues with monitoring image datafrom a camera, which may be a rear-facing camera, to determine if theface in the previously captured image appears in the camera's field ofview (operation 36). For instance, a rapid sequence of images capturedby the camera while the camera is aimed at a desired scene, are analyzedto detect when the subject has moved into the camera's field of view.This may be accomplished using suitable facial recognition software.Facial recognition algorithms with accuracy sufficient for the purposesof this invention are widely available. For example, facial recognitionis included in the iPhoto™ application by Apple Inc., which uses facialdetection to identify faces of people in photographs and facialrecognition to match faces that look like the same person. When thesubject has been recognized in the camera's field of view, a countdownmay be initiated at 38 to give the photographer (camera user) and/or thesubject additional time to compose the scene before a picture of thescene is taken. This may be implemented using a variable timer that maybe set in advance by the photographer. The photographer and/or thesubject may be alerted to the countdown with a sequence of beeps orlight flashes. At the end of the countdown, the image is captured at 40,i.e. the picture is taken.

An image capture process in accordance with another embodiment of theinvention is shown in FIG. 5. This process is similar to the processpreviously described and is begun with an appropriate user selection at50. The subject's image is captured at 52 and verified at 54 in the samemanner described above. Unlike the previously described embodiment, thesubsequent search for the subject's face within the camera's field ofview is initiated manually by the camera user. For instance, if thedevice 10 has an accelerometer or other sensor that can detect movementof the device 10, the facial recognition search may be initiated by orin response to a sudden movement of the device, such as imparted by asimple flip of the camera user's hand, which is detected at 56. Whensuch movement is detected, the process proceeds at 58 to search for thesubject's face within the camera's field of view in the same manner asdescribed above. The image is then captured at 60.

In accordance with another embodiment of the invention, a portableelectronic device has a camera function implemented by a cameracontroller, to conduct a process for taking a picture of a scene. Oncethat process is initiated (e.g., through user input via a menuselection), the controller causes the capture of an image of a subjectand then analyzes the image to detect facial features of the subjecttherein. Next, after the presence of facial features has been confirmed,the controller causes the capture of several images of the scene (e.g.,a continuous shooting sequence), and then analyzes these scene imagesusing facial recognition to detect the subject's face therein. Inparticular, the previously captured image of the subject may be analyzedto extract certain facial features from it, and these are then comparedusing a facial recognition algorithm to facial features extracted fromthe scene images, until a match is detected. This means that the subjecthas now moved into the scene. In response, the controller causes asubsequent capture of an image of the scene, which is then stored in thedevice as the desired picture.

The camera function may use a single image sensor array to which thecontroller is coupled, to cause the capture of all of the images duringthe picture taking process. Alternatively, multiple sensor arrays may beused, e.g. a first image sensor array to to capture the subject's image,and a second, different image sensor array to capture the scene imagesand the subsequent scene image. In one embodiment, the first and secondsensor arrays are aimed in substantially opposite directions.

The camera controller may be programmed to signal an alert to a user ofthe device in response to having detected facial features of the subjectin the subject's image. This alert may be text on a display screen ofthe device, indicating to the user that the subject should now move intothe scene for the picture taking process to continue.

The camera controller may also be programmed to begin a countdown inresponse to having detected the subject's face. Upon finishing thecountdown, it signals the subsequent capture of an image of the scene,where at this point the scene has most likely been finalized and thesubject is likely present in the scene.

In one embodiment, the controller can conduct the entire picture takingprocess without any user input, once the process has been initiated. Inother embodiments, the controller can prompt the user before takingaction. For instance, the user may be prompted to accept the detectedfacial features of the subject, before the controller signals thecapture of the sequence of scene images. In yet another embodiment, oncethe controller signals that the scene images are to be capture, itautomatically begins to analyze these scene images as they are beingproduced (without user input being required) and then automaticallybegins the countdown once the subject's face has been detected.

In yet another embodiment, the device has a motion sensor to which thecontroller is coupled to sense movement of the device. The controller isto analyze the scene images using facial recognition, to detect thesubject's face therein, upon sensing movement of the device (e.g., thedevice being flipped or rotated.)

It will be apparent from this description that aspects of the presentinvention may be embodied, at least in part, in software, hardware,firmware, or in combination thereof. That is, the techniques may becarried out in a computer system or other data processing system inresponse to its processor, such as a microprocessor, executing sequencesof instructions contained in a memory, such as ROM, volatile RAM,non-volatile memory, cache, or a remote storage device. In variousembodiments, hardwired circuitry may be used in combination withsoftware instructions to implement the present invention. Thus, thetechniques are not limited to any specific combination of hardwarecircuitry and software or to any particular source for the instructionsexecuted by the data processing system. In addition, throughout thisdescription, various functions and operations are described as beingperformed by or caused by software code to simplify description.However, those skilled in the art will recognize that what is meant bysuch expressions is that the functions result from execution of code bya processor, such as a microprocessor.

A machine readable medium can be used to store software and data whichwhen executed by the data processing system causes the system to performvarious methods of the present invention. This executable software anddata may be stored in various places including for example ROM, volatileRAM, non-volatile memory, and/or cache. Portions of this software and/ordata may be stored in any one of these storage devices.

The invention also relates to apparatus for performing the operationsherein. This apparatus may be specially constructed for the requiredpurposes, or it may comprise a general purpose computer selectivelyactivated or reconfigured by a computer program stored in the computer.Such a computer program may be stored or transmitted in amachine-readable medium. A machine readable medium includes anymechanism that provides (i.e., stores and/or transmits) information in aform accessible by a machine (e.g., a computer, network device, personaldigital assistant, manufacturing tool, any device with a set of one ormore processors, etc.). For example, a machine readable medium includesrecordable/non-recordable media such as, but not limited to, amachine-readable storage medium (e.g., any type of disk including floppydisks, optical disks, CD-ROMs, and magnetic-optical disks, read-onlymemories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, flashmemory, magnetic or optical cards, or any type of media suitable forstoring electronic instructions), or a machine-readable transmissionmedium such as, but not limited to, any type of electrical, optical,acoustical or other form of propagated signals (e.g., carrier waves,infrared signals, digital signals, etc.).

Additionally, it will be understood that the various embodimentsdescribed herein may be implemented with a variety of data processingsystems. For example, such data processing systems may be a cellulartelephone or a personal digital assistant (PDA) or an entertainmentsystem or a media player (e.g., an iPod) or a consumer electronicdevice, etc., each of which can be used to implement one or more of theembodiments of the invention.

Throughout the foregoing specification, references to “one embodiment,”“an embodiment,” “an example embodiment,” etc., indicate that theembodiment described may include a particular feature, structure, orcharacteristic, but every embodiment may not necessarily include theparticular feature, structure, or characteristic. Moreover, such phrasesare not necessarily referring to the same embodiment. When a particularfeature, structure, or characteristic is described in connection with anembodiment, it is submitted that it is within the knowledge of oneskilled in the art to bring about such a feature, structure, orcharacteristic in connection with other embodiments whether or notexplicitly described. Various changes may be made in the structure andembodiments shown herein without departing from the principles of theinvention. Further, features of the embodiments shown in various figuresmay be employed in combination with embodiments shown in other figures.

In the description as set forth above and claims, the terms “coupled”and “connected,” along with their derivatives, may be used. It should beunderstood that these terms are not intended to be synonymous with eachother. Rather, in particular embodiments, “connected” is used toindicate that two or more elements are in direct physical or electricalcontact with each other. “Coupled” may mean that two or more elementsare in direct physical or electrical contact. However, “coupled” mayalso mean that two or more elements are not in direct contact with eachother, but yet still co-operate or interact with each other.

Some portions of the detailed description as set forth above arepresented in terms of algorithms and symbolic representations ofoperations on data bits within a computer memory. These algorithmicdescriptions and representations are the means used by those skilled inthe data processing arts to most effectively convey the substance oftheir work to others skilled in the art. An algorithm is here, andgenerally, conceived to be a self-consistent sequence of operationsleading to a desired result. The operations are those requiring physicalmanipulations of physical quantities. Usually, though not necessarily,these quantities take the form of electrical or magnetic signals capableof being stored, transferred, combined, compared, and otherwisemanipulated. It has proven convenient at times, principally for reasonsof common usage, to refer to these signals as bits, values, elements,symbols, characters, terms, numbers, or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the discussion as setforth above, it is appreciated that throughout the description,discussions utilizing terms such as “processing” or “computing” or“calculating” or “determining” or “displaying” or the like, refer to theaction and processes of a computer system, or similar electroniccomputing device, that manipulates and transforms data represented asphysical (electronic) quantities within the computer system's registersand memories into other data similarly represented as physicalquantities within the computer system memories or registers or othersuch information storage, transmission or display devices.

The invention also relates to apparatus for performing the operationsherein. This apparatus may be specially constructed for the requiredpurposes, or it may comprise a general purpose computer selectivelyactivated or reconfigured by a computer program stored in the computer.Such a computer program may be stored or transmitted in amachine-readable medium, such as, but is not limited to, amachine-readable storage medium (e.g., any type of disk including floppydisks, optical disks, CD-ROMs, and magnetic-optical disks, read-onlymemories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, flashmemory, magnetic or optical cards, or any type of media suitable forstoring electronic instructions), or a machine-readable transmissionmedium such as, but not limited to, any type of electrical, optical,acoustical or other form of propagated signals (e.g., carrier waves,infrared signals, digital signals, etc.).

The algorithms and displays presented herein are not inherently relatedto any particular computer system or other apparatus. Various generalpurpose systems may be used with programs in accordance with theteachings herein, or it may prove convenient to construct morespecialized apparatuses to perform the method operations. The structurefor a variety of these systems appears from the description above. Inaddition, the invention is not described with reference to anyparticular programming language. It will be appreciated that a varietyof programming languages may be used to implement the teachings of theinvention as described herein.

Embodiments of the invention may include various operations as set forthabove or fewer operations or more operations or operations in an orderthat is different from the order described herein. The operations may beembodied in machine-executable instructions that cause a general-purposeor special-purpose processor to perform certain operations.Alternatively, these operations may be performed by specific hardwarecomponents that contain hardwired logic for performing the operations,or by any combination of programmed computer components and customhardware components.

Throughout the foregoing description, for the purposes of explanation,numerous specific details were set forth in order to provide a thoroughunderstanding of the invention. It will be apparent, however, to oneskilled in the art that the invention may be practiced without some ofthese specific details. Accordingly, the scope and spirit of theinvention should be judged in terms of the claims that follow as well asthe legal equivalents thereof.

1. In a portable electronic device having at least one camera, a methodcomprising automating image capture by the camera, the automated imagecapture triggered by detection of a subject in the camera's field ofview matching an image of the subject previously captured by the camera.2. The method of claim 1 wherein the image previously captured by thecamera comprises a human face.
 3. The method of claim 2 whereindetection of a subject in the camera's field of view matching an imagepreviously captured by the camera utilizes facial recognition.
 4. Themethod of claim 1 wherein the detection of a subject is triggered bysensing movement of the device.
 5. In a portable electronic devicehaving first and second cameras on opposing sides of the device, amethod comprising automating image capture by the first camera, theautomated image capture triggered by detection of an image in the firstcamera's field of view matching an image previously captured by thesecond camera.
 6. The method of claim 5 wherein the image captured bythe second camera comprises an image of a human face.
 7. The method ofclaim 6 wherein detection of an image in the first camera's field ofview matching an image previously captured by the second camera utilizesfacial recognition.
 8. The method of claim 5 wherein the first camera isa rear-facing camera and the second camera is a front-facing camera. 9.The method of claim 5 wherein the automated image capture by the firstcamera is further triggered by sensing movement of the device.
 10. Aportable electronic device having a camera function, comprising: acamera controller to conduct a process for taking a picture of a scene,in which the controller causes capture of an image of a subject and thenanalyzes the image to detect facial features of the subject therein,causes capture of a plurality of images of the scene and then analyzesthe scene images using facial recognition to detect the subject's facetherein and in response causes a subsequent capture of an image of thescene to be stored in the device as said picture.
 11. The portableelectronic device of claim 10 further comprising an image sensor arrayto which the controller is coupled to cause the capture of all of saidimages.
 12. The portable electronic device of claim 10 furthercomprising: a first image sensor array to which the camera controller iscoupled to capture the subject's image: and a second, different imagesensor array to which the camera controller is coupled to capture theplurality of scene images and the subsequent scene image, wherein thefirst and second sensor arrays are aimed in substantially oppositedirections.
 13. The portable electronic device of claim 10 wherein thecamera controller is to signal an alert to a user of the device inresponse to having detected facial features of the subject in thesubject's image, wherein the alert indicates to the user that thesubject may now move into the scene.
 14. The portable electronic deviceof claim 10 wherein the camera controller begins a countdown in responseto having detected the subject's face, and upon finishing the countdownsignals the subsequent capture of an image of the scene.
 15. Theportable electronic device of claim 10 wherein the controller is toconduct said picture taking process without any user input, once theprocess has been initiated.
 16. The portable electronic device of claim10 further comprising: a motion sensor to which the controller iscoupled to sense movement of the device, wherein the controller is toanalyze the scene images using facial recognition, to detect thesubject's face therein, upon sensing movement of the device.
 17. Anarticle of manufacturing comprising: a non-transitory machine-readablestorage medium having stored therein instructions that program aprocessor, the processor being a component of a portable electronicdevice having a camera, to cause a capture of an image of a subject, toanalyze a facial feature of the subject in the captured image, and tomonitor subsequent image data to determine when the subject appears inthe image data, and to cause a subsequent capture of an image of a scenein response to the subject appearing in the image data.
 18. The articleof manufacture of claim 17 wherein the instructions cause the image ofthe subject to be captured by a front-facing camera having an imaginglens disposed on the same surface of the device as a digital viewfinder.19. The article of manufacture of claim 18 wherein the instructionscause the subsequent image data and the image of the scene to becaptured by a rear-facing camera having an imaging lens disposed on theopposing surface of the device as the digital viewfinder.
 20. Thearticle of manufacture of claim 17 wherein the machine-readable storagemedium contains additional instructions that program the processor toinitiate monitoring of the subsequent image data in response to sensingmovement of the device.
 21. The article of manufacture of claim 17wherein the machine-readable storage medium contains additionalinstructions that program the processor to initiate monitoring of thesubsequent image data in response to verifying the existence of facialfeatures in the captured image.
 22. The article of manufacture of claim17 wherein the machine-readable storage medium contains additionalinstructions that program the processor to start a countdown when thesubject has been recognized in the subsequent image data, signal analert for the countdown, and at the end of the countdown cause thesubsequent capture.